Method for defect detection for rolling elements

ABSTRACT

A method of detecting defects in a rolling element is provided herein. The method includes collecting visual data or information via a microscope assembly. This visual data is then processed and algorithms, filters, or other analytical tools or engines are applied to detect if any defects are present on the rolling element. The method includes automatically flagging or identifying any defects, and this step is then checked or verified by a user or different entity. Depending on input from the user, the defects can either be confirmed or can be identified as a false detection. Information from the user&#39;s decision-making process is then fed back into the system, processors, algorithms or other analytic aspects of the system. This information is then used to improve the accuracy of the detection algorithms. This disclosure provides an automated system and process for more efficiently and reliably identifying defects on rolling elements.

FIELD OF INVENTION

The present disclosure relates to an inspection method for analyzingrolling elements.

BACKGROUND

Rolling elements, such as ceramic balls and steel balls, are used in awide range of different industrial applications and settings. It isimportant to inspect these rolling elements prior to installation or useto identify any defects, such as surface defects. In some high precisionapplications, it is important to use rigorous inspection methods thatanalyze the entire surface of the rolling element. Known methods forperforming this analysis include using stereo microscopic analysis andmanual labor processes that include visually identifying the rollingelements for defects. This process can suffer from multipledeficiencies. For instance, this inspection process requires constantadjustment of lighting conditions provided by an external source to thestereo microscope. Defect identification also depends on humanexperience and subjective judgement, which inherently varies betweenindividuals. This process therefore can encounter issues regarding humanbias or inadequate training, and therefore lead to high error rates inthe inspection process.

One common type of rolling element includes a spherical ball bearing,which can be difficult to photograph or image due to its geometry. Therolling elements must be rotated manually under the stereo microscope toobtain a 360 degree view of the ball. This rotation does not reliablycover 100% of the surface area due to variations in manual rotationprocess. It is desirable to detect defects such as scratches, cuts,missing sections, rust etc. at the micron level, and this makes thecurrent manual approach challenging.

These processes lead to unidentified defects or false defect detections.Scaling the inspection system for handling large volumes of rollingelements requires costs, including infrastructure and labor.

It would be desirable to provide an automated inspection process andworkflow that reliably identifies defects in a relatively short timeperiod.

SUMMARY

A method of detecting defects in a rolling element is provided herein.The method generally includes arranging a rolling element in amicroscope assembly. The microscope assembly is configured to scan anouter surface of the rolling element to obtain a plurality of surfaceimages of the rolling element. The method includes associating each ofthe plurality of surface images with a specific region of the rollingelement. The plurality of surface images are configured to be stored ina database. The method further includes identifying defects based on theplurality of surface images via a processing module using an initial setof analysis parameters. The method includes generating a plurality ofinteractive inspection icons associated with the plurality of surfaceimages that each corresponds to a unique geographic region of therolling element. The method includes displaying the plurality ofinteractive inspection icons via a user interface. The user interfaceincludes an input interface including an approve option, such as aninterface tool, button, etc., and a rejection option, such as aninterface tool, button, etc. The method includes storing decisionalinformation regarding selection of the approve option and the rejectionoption from the input interface in the database. Finally, the methodincludes updating the initial set of analysis parameters based on thedecisional information.

The method can further include providing the plurality of surface imagesto the database in real time. The method can also include identifyingthe defects on the rolling element via a processor or AI engine that isconfigured to identify defects based on a predetermined set of analysisparameters.

The user interface can be configured to display a two-dimensional imageassociated with the plurality of surface images. In another embodiment,the user interface is configured to display a three-dimensional image ofthe plurality of surface images.

The plurality of interactive inspection icons can be generated anddisplayed in real-time while the microscope assembly scans the rollingelement. The method can also include identifying at least one of: (i) adefect type on the outer surface of the rolling element, or (ii) a sizeof a defect on the outer surface of the rolling element.

The plurality of interactive inspection icons can include at least oneindicia that is representative of a defect condition. In one aspect, theindicia is configured to correspond to a level of severity of anydefects on the outer surface of the rolling element in a specificgeographic region.

At least one visual characteristic of the plurality of interactiveinspection icons can be configured to modified based on the level ofseverity of any defects that are automatically detected.

The method can also include generating and displaying additionalinformation regarding the corresponding geographic region of the rollingelement via engagement of the plurality of interactive inspection iconson the user interface. For example, a user can click a cursor or mouseicon on interactive inspection icons to access additional information ordata.

The method can also include displaying at least one image of thecorresponding geographic region of the rolling element based onengagement of the plurality of interactive inspection icons on the userinterface.

Additionally, the method can include generating a report including thesurface images and the decisional information. This report can then beused for future analysis.

Additional embodiments are disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing Summary and the following Detailed Description will bebetter understood when read in conjunction with the appended drawings,which illustrate a preferred embodiment of the disclosure. In thedrawings:

FIG. 1 is a work process flow diagram for a defect detection process ormethod.

FIG. 2A is a perspective view of a microscope assembly for analyzingrolling elements.

FIG. 2B is a front view of the microscope assembly of FIG. 2A.

FIG. 2C is a side view of another aspect of the microscope assembly ofFIGS. 2A and 2B.

FIG. 2D is another side view of the microscope assembly.

FIG. 2E illustrates a plurality of holder assemblies configured tosupport rolling elements.

FIG. 2F illustrates the plurality of holder assemblies engaging aplurality of rolling elements.

FIG. 3A is an illustration of grid inspection data according to oneaspect.

FIG. 3B is a visual representation of rolling element inspection grids.

FIG. 4 illustrates an exemplary workflow for one aspect of analyzing arolling element.

FIG. 5A is a view of a user interface according to one aspect.

FIG. 5B is a view of a user interface according to another aspect.

FIG. 5C is an illustration of one portion of a rolling element surfaceincluding a defect.

FIG. 5D is an illustration of one portion of a rolling element surfacelacking any defects.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Certain terminology is used in the following description for convenienceonly and is not limiting. “Axially” refers to a direction along an axis(X) of an assembly. “Radially” refers to a direction inward and outwardfrom the axis (X) of the assembly.

A reference to a list of items that are cited as “at least one of a, b,or c” (where a, b, and c represent the items being listed) means anysingle one of the items a, b, or c, or combinations thereof. Theterminology includes the words specifically noted above, derivativesthereof and words of similar import.

The process shown in FIG. 1 provides an arrangement that fully automatesand provides digitalization of workflow combining confocal microscopy,Artificial Intelligence (AI) deep learning vision algorithms,interactive web application, and a connected database for fulltraceability covering near 100% surface area and near 100% defectiondetection. In one aspect, 100% surface analysis is provided by thepresent disclosure. This analysis can be ensured via overlappingadjacent areas or regions that are scanned and overlaying these imagesor data associated with these images with each other. In one aspect, thesystem comprises a confocal microscopy component, real time dataacquisition and streaming component, AI imaging analytics enginecomponent, internet application component, and a cloud infrastructurecomponent for database streaming connectivity.

As shown in FIG. 1 , an inspection module 100 is provided. Theinspection module 100 can include a microscope 105, such as a confocalmicroscope. In one aspect, the confocal microscope can include a MahrMarSurf CM Explorer confocal microscope. One of ordinary skill in theart would understand that other types of automatic confocal systemscould be used, such as Zeiss SmartProof, or any system that utilizes anautomatic positioning or rotating stage. The microscope software can besynced to an automatic stage and can be configured to automatically jogor move and capture surface scanning information of the rolling element.

One of ordinary skill in the art would understand that the inspectionmodule 100 can be configured to analyze rolling elements, such as ballbearings or rollers, or any other type of bearing component, or othersurfaces for elements. More specifically, a first computer processingunit (CPU) 110 can be provided that is electronically connected, eitherdirectly or wirelessly, to the microscope 105. The first CPU 110 caninclude an interface for receiving and transmitting data, a memory unit,a user interface, a monitor or display, a processor, and other knownelectronic components. In one aspect, the first CPU 110 can comprise aMahr CM Select Edge computer. One of ordinary skill in the art wouldunderstand that other computer systems, such as systems commerciallyavailable from HP, Dell, or other manufacturers could be used. One ofordinary skill in the art would understand that other computing unitsand processors could be used.

The first CPU 110 can be configured to generate a three-dimensional viewof the surface area of the rolling element, and can provide surface scaninformation at the micron level. The first CPU 110 can be configured toadjust optical parameters, such as lighting and focus. In one aspect,software or other interface tools can be implemented that allowoptimization of magnification of the lens to the surface area coverageof the sample. For example, a range of magnification can be provided bythe system that allows anywhere from 5× the magnification to at least100× the magnification, as well as any intermediary values. Through auser interface, such as element 305, metadata can be used to configurethe first CPU 110 and the microscope 105. Input information, such assize input, material type, rolling element type, etc. can be entered viathe user interface 305.

The first CPU 110 can be configured to automatically scan the rollingelement and capture imaging data. In one aspect, the first CPU 110 isconfigured to drive a positioning stage, such as an X-Y stage platformor mobile stage platform. Positioning of the stage can be configured orprogrammed to begin at a first grid or row, and then move from left toright. After reaching the end of the first row, the stage can beconfigured to move to the next row, and again move left to right. Thisprocess can be repeated until reaching the end of the sample or rollingelement. In one aspect, the imaging data is obtained in a grid pattern.One of ordinary skill in the art would understand that other opticalscanning patterns could be used. This imaging data can then betransmitted instantly and in real-time for further processing.

The imaging data can be transmitted from the inspection module 100 to acomputing module 200, which can include at least a database 205 (i.e. adata storage unit) and a processing module 210 (i.e. an imaginganalytics and processing unit). The database 205 can be configured toreceive and transmit data to and from the inspection module 100. Thedatabase 205 can include any combination of any commercially availablestructured databases (such as SQL) and unstructured databases (such asMongoDB). The database 205 can be configured to store and track metadatainformation, such as testing conditions, traceability of the rollingelements tested, operator interactions etc. In another aspect, thedatabase 205 can be configured to store 2D flattened images or 3Dsurface scan measurements.

In one aspect, the database 205 can be directly connected to theinspection module 100, or can be wirelessly connected to the inspectionmodule 100. Information, signals, and data from the microscope 105 canbe stored in the database 205, such as via a cloud-based or anon-premise data collection and storage configuration. The microscope 105can be configured to transmit or stream multi-dimensional informationabout each grid that is analyzed of the rolling element. In one aspect,this information is stored in a Flexible Image Transport System (FITS)format. One of ordinary skill in the art would understand that theformat of this data and information can vary.

In one aspect, custom data pathways or pipelines can be configured tointegrate and implement tools from third party cloud computingresources, such as Microsoft Azure, Amazon Web Services (AWS), GoogleCloud, etc., for data transfer between the microscope 105 and thedatabase 205. Data can be shared or transferred by network foldersharing for on premise databases and storages, microservices configuredto stream data to the cloud, or data pipelines to monitor dataavailability and streaming uploads.

In one aspect, the database 205 is configured to store reports and otherdata logs, such as defect reports, and also log and track informationregarding human interactions and decisions. This information is storedsuch that decisions and information can be traced or logged foradditional analysis. Additionally, the database 205 stores informationregarding algorithm performances for model monitoring and retrainingpurposes.

The user interface 305 can be configured to display the status of eachgrid that is inspected with a color status to indicate a defect wasfound. A user can agree with the identification of a defect, or overrideit as a false positive detection. Similarly, if the system failed toidentify a defective grid, a user can override this decision as a falsenegative. The user interface 305 and the database 205 can be configuredto log or record all instances of true positive, true negatives, falsepositives and false negatives. This information is subsequently used fortraining the system to improve the defect identification accuracy.

The processing module 210 receives information from the microscope 105.In one aspect, this information is provided in the FITS file format.This information includes data regarding focal parameters of themicroscope, field of view, optical parameters (such as intensity ofreflected light from the surface), focal area of interest capturedduring rotation, among other data.

The processing module 210 can include various aspects, such as apre-processing engine 210 a, an AI vision solution engine 210 b, and apost-processing engine 210 c. The pre-processing engine 210 a can beconfigured to handle the multi-dimensional data stream from themicroscope, such as in a grid by grid format. The pre-processing engine210 a can be configured to apply mathematical transformation (such as aleast square fit function) to translate three-dimensional surface scandata to two-dimensional color-coded spectral images. The pre-processingengine 210 a can also be configured to apply image processingtechniques, such as noise removals, segmentation for removing unwantedregions, etc. The AI vision solution engine 210 b can be configured toidentify defects for the pre-processed images. For example, in oneaspect, different types of surface defects samples can be identified byusers of the system and can be marked with bounding box locations withthe defect type on a 2D color mapped surface image. These images, alongwith accompanying user comments or annotations, can be used forconfiguring or training the AI vision solution engine 210 b.

In one aspect, the processing module 210 is configured to receivethree-dimensional imaging data and information from the microscope 105via the CPU 110 and database 205. Once this information is received bythe processing module 210, the processing module 210 can process theinformation such that any three-dimensional imaging data or informationis converted to a two-dimensional format. This process can includeflattening or converting the data or information. For example, the datacan be “flattened” or converted such that any three-dimensionalinformation or data is visually represented in two dimensions. Thisprocess can include producing or generating the images shown in FIGS. 5Cand 5D. Referring to FIG. 5C, any scratches or defects can be shown witha varying color or characteristics instead of a depth in threedimensions.

Alternatively, the imaging data from the microscope 105 can also beprocessed such that it remains in a three-dimensional format. Additionalprocessing modules or software can be implemented to convert or processthe data from the microscope 105.

The pre-processing engine 210 a can generally be configured to receive amulti-dimensional FITS file or other data from the microscope. Thepre-processing engine 210 a can synthesize, organize, or otherwiseprocess this information and convert or process the information into atwo-dimensional format. By converting this data or information into atwo-dimensional file, a deep learning algorithm can then further handleor process the information or data. The pre-processing engine 210 a canbe configured to run or perform a series of algorithms or processes thatare configured to remove noise (i.e. noise removal modules), segment thedata for region of interest (i.e. segmentation for region of interestmodules), drift removal, surface correction, and transformation fromcurved surfaces to flattened surfaces. Such algorithms or processes canuse curve fitting techniques, such as least squares fit, Mel-Penrosepseudo inverse solutions, or other conversion or processing techniques.In one aspect, convex optimization and linear algebra methodologies canbe used.

The AI vision solution engine 210 b can be used to further process thedata, for example via a transfer learning process. In one aspect, aR-CNN architecture can be implemented by this engine. The deep learningmodule can also use other vision architectures. For example, Resnet,InceptionNet, Mobilenet, EfficientNet, vision transformers or otherarchitectures can be implemented. These architectures contain severallayers ranging from twenty to two hundred, or greater, of connectedneurons. The AI vision solution engine 210 b is configured to bepreloaded or trained with millions of images that are available foropensource adoptions. These models or modules can be configured forsurface error detection by a method known as transfer learning. Thevision network architectures mentioned above can contain a series ofcascaded layers of neurons. These neurons are trained (i.e. loaded orotherwise pre-configured) with millions of images, such as from publiclyavailable sources, and are configured to extract and segment featuresfrom these images. By this transfer learning approach. the initialneural network informational (i.e. weights) layers are kept intact whichcaptures different features from the 2D images. The higher layers aretrained by providing defect images that are annotated by a user. Byapplying transfer learning approach, these vision models are trained tocapture different surface defects.

A post-processing engine 210 c can also be implemented. After the AIvision solution engine 210 b identifies the defective region andcommunicates that information, such as bounding box coordinates, pixellocations, or other information, the post-processing engine 210 c thenis configured to process this information. The post-processing engine210 c can be configured to transform the information into measurement ofthe defect found (such as in micron units). This information regardingthe defect can include the width and depth of the defect that was found,as well as type of defect or other information. This information is thentransmitted for further analysis and processing, for example to the userinterface module 300. In one aspect, the post-processing engine 210 c isconfigured to analyze image data or other information associated with aparticular defect and is further configured to calculate a depth, width,and other geometric information regarding that specific defect. Thisinformation can later be accessed by a user to determine the exact sizeof a specific detected defect.

The user interface module 300 can be considered a workstation. The userinterface module 300 can provide a central module for merginginformation from the analytics engine, database, microscope, etc., andallowing a user to interact with various aspects of these components.For example, the user interface module 300 can include a display that isconfigured to show surface scan information and data, associated witheach grid, or in various types of configurations. Results or processeddata from the processing module 210 can also be viewed, processed,manipulated, or otherwise engaged via the user interface module 300.Personnel can interact with the user interface module 300 to review dataand information from any of the other modules, engines, microscope,databases, etc.

In one aspect, a plot of grids can be generated for the user to view andreview on the user interface module 300. In one aspect, the userinterface module 300 can include a user interface 305, which is alsoreferred to herein as a workstation. The user interface 305 can includeany user interface or application. For example, the user interface 305can include any commercial desktop or laptop running the user interfaceas web application or Android or IOS tablet system running the userinterface as mobile applications.

A user can provide decisional feedback regarding whether to accept orreject a particular rolling element based on defects found by thesystem. This decision can be based on various information generated bythe system, such as the defect type, measurement parameters, andmeasurement predictions. Examples of measurement predictions can includetype of defect, location of the defect, length and depth estimates bythe AI engine, etc. One of ordinary skill in the art would understandthat other measurement predictions could be used.

As shown in FIG. 3B, the information regarding surface characteristicsof the rolling element from FIG. 3A, i.e. the grid of two-dimensionalinformation or data or grid inspection information, is ultimatelypresented to the user via the user interface 305. Specifically, the userinterface 305 can display a graphical representation of the grid ofsurface characteristics. In one aspect, each interactive inspection icon307 (hereinafter referred to as icon) on the display represents apredetermined surface area having a unique geographical position on therolling element. Based on the algorithm and processes disclosed herein,each icon 307 is assigned a particular characteristic indicating whethera region or area associated with that icon 307 includes a defect thatmeets a predetermined threshold or not. For example, if a specific areadoes not appear to include any defects, then the particular icon 307 canhave one characteristic or visual indicia 307′. In one aspect, theseicons can be assigned the color green, or a first hatching pattern,thereby indicating no significant defects are present. In anotheraspect, areas that appear to have a surface defect exceeding apredetermined threshold are assigned a different characteristic orvisual indicia 307″. In one aspect, these icons can be assigned thecolor red, or a second hatching pattern, thereby indicating a defect ispresent. Detection of these defects at this stage can be fullyautomated. For example, the processing module 210 can be configured tocarry out a comparative algorithm that analyzes images obtained by themicroscope 105 and compares those images to stored images of defectsthat were previously identified as being above a predeterminedthreshold. In one aspect, the predetermined threshold can be based onthe geometry (i.e. size, depth, etc.), or type of defect. In one aspect,any deviation from a smooth surface is identified as a deviation, andcan include pitting, scratches, dimples, bumps, or other surfaceimperfections.

In one aspect, the processing module 210 is configured to determine thegeometric characteristics of a surface area from the images from themicroscope 105 and is configured to measure defect parameters based onthe images. If the measurements that are calculated by the processingmodule 210 exceed a predetermined threshold (i.e. depth of defect, sizeof defect, etc.), then the processing module 210 can be configured toidentify a significant defect in the particular area or regionassociated with that image. The characteristics of the icons 307 can beupdated in real-time or dynamically as the processing module 210 iscarrying out this analysis and determinations. Therefore, the icons 307will sequentially be updated, such as filled in with the color red or afirst hatching pattern for a significant defect, or filled in with thecolor green or a second hatching pattern for no significant defectsdetected. A user can dynamically select each of the icons 307 after theicons 307 have been modified based on the analysis by the processingmodule 210. For example, if one of the icons 307 has been updated toindicate that the specific area associated with that icon 307 lacks anysignificant defects, then the user can select or click that specificicon to confirm that no significant defects are present. A cursor orother element can be provided that is configured to move based on a userengaging a mouse or touchscreen associated with the user interface.Selection of the icon is configured to access or retrieve the sourceinformation or data related to that specific icon. In one aspect,selection of the icon 307 by the user is configured to display theimages from the microscope 105 for that specific area of the rollingelement. The user can then perform further analysis of the areaassociated with that area. In the event that the user determines aspecific icon 307 that originally had an indication of no significantdefects does in fact have significant defects, then the user can rejectthe rolling element via engagement with a rejection or reject button oroption. In addition to addressing the specific rolling element that wasbeing analyzed, this process is useful because the decisionalinformation and the associated rolling element surface information isrepurposed or used by the system. In one aspect, the decisionalinformation teaches the system such that the system is adaptive orlearning based on user input. All the user decisions regarding the truepositives, true negatives, false positives, and false negatives relativeto the detections or decisions made by the system are captured andstored in the database. This information is used to further train theprocessing module 210.

The processing module 210 is configured to periodically update itsalgorithm to learn from a particular set of images and human/user inputor decisions. This updating can be performed in an offline mode, whenactive inspections are not occurring. In one aspect, the user's inputteaches or updates the algorithm such that the user's expertise onanalysis of defects can be automated via the algorithm. Alternatively,if an icon 307 includes an indication of a significant defect, a usercan also check whether such a significant defect is in fact present bychecking the associated data or information with that icon. For example,a user can review the images associated with a particular icon and maydetermine that the defects are not significant.

Decisions made by the user regarding whether to accept or reject a partor rolling element can be saved and stored by the system, such as via alocal database 310 or other data storage unit. This decisionalinformation can then be used by any aspect of the system, such as by theAI vision solution engine 210 b, to improve the system's accuracy rateswith respect to detecting defects and the severity (i.e. pass or fail)of these defects.

In one aspect, the analytics components of the system can be deployed ina cloud, such as a scalable Kubernetes cluster or in an on-premisecommercially available EDGE system, which can be scaled smaller orlarger depending on the specific requirements of a system. The AI systemcan process higher number of grids in real time and can be scaled higherby adding more computation units. This is configurable based on userneeds, including the cycle time or how fast a user requires informationor data from the system.

As shown in FIG. 1 , the process of analyzing rolling elements andgathering information regarding any associated defects can utilize aninspection module 100, computing module 200, and user interface module300. During step 10, the microscope 105 is used to analyze at least onerolling element 5, and preferably is used to analyze a plurality ofrolling elements.

FIGS. 2A-2F illustrate aspects of the microscope 105 and the associatedcomponents in further detail. As shown in these drawings, the microscope105 can include a rolling element manipulator or holder assembly 105 b.In one aspect, the holder assembly 105 b can remain stationary and floatabove an automated X-Y stage platform 105 a that is configured to rotateor roll the rolling element 5. The automated X-Y stage platform 105 acan be configured to move automatically relative to the opticalcomponent 105 c of the microscope 105 based on input signals. Theoptical component 105 c can include a lens, camera, and/or other opticalscanning equipment configured to obtain images or otherwise analyze therolling element 5. A mount 105 d, otherwise known as a bridge or frame,can be provided above the X-Y stage platform 105 a, and below theoptical component 105 c. The mount 105 d can be configured to receive orhold the holder assembly 105 b.

As shown in FIGS. 2E and 2F, multiple types and sizes of holderassemblies 105 b can be provided. FIG. 2E illustrates a plurality ofholder assemblies 106 a, 106 b, 106 c, 106 d, 106 e, 106 f. Each of theholder assemblies can include an outer frame and an interior space thatis configured to receive a rolling element. Depending on the size of therolling element to be analyzed, the interior space can vary, as shown bythe various holder assemblies 106 a-106 f. In the interior space, prongsor arms 107 a, 107 b, 107 c, 107 d, 107 e, 107 f can extend inwardly andbe configured to engage with the respective rolling elements. The holderassembly 105 b can be configured to be attached to a frame or portion ofthe microscope 105, such as a base or stationary portion. As the X-Ystage platform 105 a moves, the holder assembly 105 b remainsstationary. Due to the arms engaging the rolling element 5 on a side orperipheral surface, the rolling element 5 is configured to be turned orrotated as the X-Y stage platform 105 a moves.

During step 10, information or data from the microscope 105 istransmitted (i.e. wirelessly, wired, or through any connection) to theCPU 110. The CPU 110 is generally configured to communicate with themicroscope 105, including receiving data and information from themicroscope 105 regarding the analyzed rolling elements. The CPU 110 canbe configured to receive optical information, such as images of therolling elements, including a three-dimensional view of the surfacearea, surface scan information (i.e. at the micron level), and otherviews of the rolling element. In one aspect, the information regardingthe surface of the rolling element is generated as a two-dimensionalgrid, such as shown in FIG. 3A, via the CPU 110. In this view, each icon306 represents one specific or unique area of the rolling elementsurface. In one aspect, each icon 306 represents an area, i.e.microscopic area, of the rolling element surface. In one aspect, eachdot or circle (i.e. icon) represents an area of approximately of atleast 10× magnification. In one instance, this can correspond to an areaof 1.6 mm by 1.6 mm. One of ordinary skill in the art would understandthat various magnifications and grid sizes can be used for differenttypes of rolling elements. For example, in one aspect, at least 1,000sample regions or areas can be analyzed by the system. In a preferredembodiment, at least 1,200 sample regions or areas can be analyzed bythe system. This information, including data regarding each surface areaas represented by the icon 306, is stored in a file for furtherprocessing. The grid can be reproduced, after further processing, to auser on a user interface for further analysis.

During step 15, information or data is automatically transmitted to thedatabase 205. This information can include all imaging related dataregarding the rolling element, such as the grid shown in FIG. 3A, andimages showing the surface characteristics of the rolling elements.

During step 20, data and information from the database 205 can betransmitted to the processing module 210. The processing module 210 caninclude the pre-processing engine 210 a, the AI vision solution engine210 b, and the post-processing engine 210 c. The processing module 210can be configured to run the AI vision solution engine 210 b in a dockercontainerized environment either in an on-premise EDGE system or in adeployed state in a third-party cloud architecture, such as in aKubernetes configuration. Communication between the module 210, the userinterface 305, and the database 205 can be configured to be processed orhandled via Application Programming Interface (API) calls. Theprocessing module 210 can be configured to receive requests from theuser interface 305 regarding the inspection grid that is available forprocessing and metadata to locate it on the storage from the userinterface 305. The processing module 210 can be configured to extractstored data from the database 205 and pre-process it to atwo-dimensional surface image. The AI vision solution engine 210 b canbe configured to run a defect scan on the image and generate acollective report, and communicate this information back to the userinterface 305, such as via an API response.

During step 25, data and information can be transmitted between theprocessing module 210 and the user interface 305. An API can be providedthat allows users to interact with data and information provided via theprocessing module 210. For example, using the API, a user can viewinformation regarding the surfaces of the rolling elements. In oneaspect, this information can be presented to a user in a 3D modelshowing the rolling element, or can be provided as a grid, such as shownin FIG. 3A, with each icon representing a specific area of the rollingelement. Using this interface, the user can review characteristics ofthe rolling element. For example, a user can select (such as viaengagement with a cursor) a specific portion of the imaging data andreview the surface of a specific portion of the rolling element. Theprocessing module 210 can be configured to automatically identify (i.e.flag or otherwise highlight) specific portions of the rolling elementthat may include a defect or other imperfection. A user can then reviewthese specifically identified portions to determine whether those areasinclude defects that are sufficient to reject the rolling element. Thestandard for passing inspection can vary greatly depending on customerrequirements, industry requirements, material type, size of rollingelement, etc. For example, a user can set a predetermined geometrycharacteristic which, if triggered, indicates that the particular defectis impermissibly high to allow the rolling element to proceed forfurther use. In one aspect, the user interface provides a plurality ofparameters, such as geometrical characteristics, which can be adjustedto have a higher or lower sensitivity for a particular type ofapplication or rolling element.

During step 30, the process determines whether defects have been found.This process can depend on the user diagnosing whether the surfaceimperfections or defects found by the processing module 210 aresufficient to qualify as fatal or severe defects (i.e. defects of such amagnitude as to reject the rolling element for use).

In one aspect, the user interface provides an approve/reject feature,option, interface, or button that allows a user to manually selectwhether to approve or reject a specific rolling element based on thesurface defects that were identified. The process is configured to storeinformation regarding whether to accept or reject a specific componentalong with the surface characteristic information. A feedback loop orstep 40 is provided in which if a user rejects a rolling element, thenthat information is transmitted to CPU 110, i.e. the processor computerin communication with the microscope 105. This information can act astrigger for stopping further data collection because the user hasidentified the defects and decided to reject the rolling elements.

Step 45 in FIG. 1 represents an analysis step in which informationregarding a rejected rolling element is generated and collected, alongwith the raw or initial surface characteristic data. A report, such as adefect report, can be generated using these parameters that can then bestored in database 310. This procedure provides a fully traceableconfiguration in which all decisions and defects associated with aparticular rolling element or batch of rolling elements can be storedfor further review and subsequent analysis. This information can befurther processed in order to further improve the accuracy of thesystem. The database 310 can be connected to the processing module 210,and the user interface 305 such that all data or information stored ondatabase 310 can be transmitted or accessed by these components.

As shown in FIG. 1 , if the defect is not sufficient to reject therolling element, then step 50 shows that this data and information isalso stored on database 310.

FIG. 4 illustrates an exemplary workflow 400. As shown in FIG. 4 , datais acquired at step 410. In this step, the data can include surfacecharacteristics of the rolling element, as well as geographic locationsof said surface characteristics. This information, which can includedata set coordination information and locational information, can bestored in a grid pattern, two-dimensional data array, orthree-dimensional mapping image. The data in step 410 can be acquiredusing a confocal microscope assembly, as described herein with respectto the inspection module 100, and its associated components.

Step 420 includes processing of the data from step 410. During thisstep, the data can be pre-processed. This step can include applyingfilters to remove noise or other noise removal techniques, segmentationfor region of interest modules, drift removal, surface correction, andtransformation from curved surface data to flattened surface data. Thethree-dimensional surface scan information can be mapped intotwo-dimensional images representing the surface in terms of colormappings. For example, the color red can indicate a deviation in anegative direction (i.e. scratch) from a curved outer surface, and thecolor blue can indicate a deviation in a positive direction (i.e. bump)from a curved outer surface.

Step 430 can include processing of the two-dimensional color mappedsurface scan images. In one aspect, this step is carried out via theprocessing module 210, and more specifically through the AI visionsolution engine 210 b. This step can also be configured to identifydifferent types of defects. Step 430 can be configured to provideinformation about the defect type, location, depth, length, and level ofconfidence in its prediction to the user interface. In one aspect, alevel of confidence is based on the system previously encountering oranalyzing similar defects as compared to the currently or presentlyencountered defect. If a particular defect appears similar in appearanceas a defect that has already been encountered by the system, then thelevel of confidence is relatively higher as compared to a defect that isnot similar to previously encountered defects.

Step 440 includes displaying the data to a user via a user interface.This step can include providing a visual mapping of the rolling elementand showing the surface characteristic data via a plurality of methods.For example, a model of the rolling element can be generated for a userto manipulate, rotate, or otherwise move in order to view variousregions of the rolling element. In one aspect, a grid pattern of all ofthe regions of the rolling element can be generated. Representativeicons can be used to show the various regions of the rolling element. Auser can select, such as by clicking a cursor, over a specific region orarea of interest on the user interface display. Further information oranalysis can be provided to the user during this step, such as specificparameters of the surface defect. All the defect information metadatafrom the AI vision solution engine 210 b is provided to the user in agraphical user interface. For example, the size or depth of the defecton the rolling element can be provided to the user.

Step 450 includes an approval or rejection determination. The userreviews the defect metadata predictions generated by the AI visionsolution engine 210 b from previous steps and then makes the decisionabout rejecting or accepting the rolling element. This determination canbe selected by a user via the user interface. For example, the userinterface can display an “approve” or similar button or function, and a“reject” or similar button or function. The user can select to approveor reject a specific rolling element based on the severity, type, orother aspect of the surface defect. For example, if the surface defectis unacceptably large or severe, then the user may select to reject therolling element. The specific parameters for accepting or rejecting aspecific rolling element may be determined by the class or type ofrolling element, and by the desired application or field of use. Forexample, for high precision applications, the tolerance or acceptabilityfor defects can be much lower than less sensitive or criticalapplications.

Step 460 includes associating or linking the decisional data from step440 with the original surface characteristic data from step 410. Forexample, if a surface defect of a first predetermined size was detectedin step 410, then step 450 associates that particular defect and thefirst predetermined size with the decision to either accept or rejectthe rolling element from step 440. All the information from step 420 to440 are synced and stored in a database 310. This information can beused, for example, during offline training, to continuously update andimprove the accuracy of the processing module 210 in multiple steps,such as steps 420-440.

A feedback loop of the data from step 470, including the determinationto approve or reject a rolling element and the associated surfacecharacteristic data, is then provided back to step 410. Once the userinputs a decision to reject the rolling element, the process can stopand additional information, such as grid data, does not need to becollected. The feedback from step 470 is then transmitted or fed back tostep 410, to stop any ongoing real time data collection for the givenrolling element.

FIGS. 5A, 5B, 5C, and 5D provide further details of the user interface,which can be provided on the user interface 305 or user interface, suchas laptops, computers, smartphones or tablet devices. As shown in FIG.5A, an interface is displayed on a monitor, such as a computer screen orother electronic device. Various functionalities can be implemented onthe user interface. A “view data” button 510 can be provided whichallows a user to access the data associated with a rolling element, suchas surface data collection status, metadata information, two-dimensionalsurface scan information, defect identification status, etc. Additionalbuttons can also be provided on the interface. For example, a stopanalysis button 520 can be provided that allows a user to stop theanalysis process. This button can be configured to restart the analysisas well. These buttons can use API calls for communicating with the userinterface 305 and also to any of the databases. A dashboard button 515can also be provided that allows a user to access a summary report ofall the inspections conducted in the given period and correspondingdefect status.

On the screen, multiple interactive inspection icons 507 are displayed.Each of these interactive inspection icons are associated with aspecific area of a rolling element and can be considered ascorresponding to inspection grids. Each of these interactive inspectionicons 507 can have some indicia, such as color or hatching, to indicatetheir particular status. For example, the color green can be used toindicate that no defects were identified by the system, while the colorred can be used to indicate that severe defects were identified by thesystem. Additionally, another color, such as orange or yellow, can beused to indicate that a particular inspection grid requires userattention or input for further analysis. In one aspect, each interactiveinspection icons 507 is initially gray, white, or otherwise non-coloredif no analysis is available for that particular inspection grid. As theinformation or data is collected (i.e. images of the rolling element arecaptured and analyzed), the interactive inspection icons 507 begin tochange or update. For example, the interactive inspection icons 507 canturn green if analysis has been carried out and no significant defectswere detected by the algorithms and analysis methods. In another aspect,the interactive inspection icons 507 can turn red if a significantsurface defect is detected. The interactive inspection icons 507 canchange in real time or dynamically as the analysis is carried out. Eachinteractive inspection icon 507 can be adjusted to show an appropriatestatus in sequential order. A user can therefore review the status ofthe rolling element analysis in real time and simultaneously analyze therolling element inspection grids. For example, if the processing module210 or other analysis component of the system detects a defect andsignifies this via changing the color to red, or a first hatchingpattern, for a specific interactive inspection icon 507, then the usercan click that specific icon 507″ and the system is configured to thendisplay information regarding that specific inspection grid. In oneaspect, the icon 507″ can be associated with an inspection grid or areaof the rolling element as shown in FIG. 5C. As shown in FIG. 5C, theuser interface is configured to display the image for specificinspection grids. A defect (D) is shown in FIG. 5C for that specificinspection grid. Further information can be shown on the user interface,including a grid ID, grid number, status, defect type, pixel informationor location, and length of the defect.

The processing module 210 can be configured to measure aspects of thedefect based on pixel information and analysis. The user can review thisimage data and information to determine whether the icon 507″ associatedwith that particular area has a significant defect. In one example, thisallows the user to verify the decision made by the processing module210. The user can additionally access image data associated withinteractive inspection icon 507 that were identified as not having anydefects, such as icon 507′. Clicking or engaging icon 507′ can accessadditional information, images, or data, such as the informationdisplayed by FIG. 5D. FIG. 5D shows image data or information associatedwith the icon 507′. As shown in FIG. 5D, the image for that particularinspection grid does not include any significant defects or issues, ascompared to FIG. 5C.

The system and processes disclosed herein address costs regarding fixedinvestment resources that are typically required for inspection ofrolling elements, and also provides an improved scalable configurationthat will improve over time based on deep learning concepts.

Having thus described the present disclosure in detail, it is to beappreciated and will be apparent to those skilled in the art that manyphysical changes, only a few of which are exemplified in the detaileddescription of the invention, could be made without altering theinventive concepts and principles embodied therein.

It is also to be appreciated that numerous embodiments incorporatingonly part of the preferred embodiment are possible which do not alter,with respect to those parts, the inventive concepts and principlesembodied therein.

The present embodiment and optional configurations are therefore to beconsidered in all respects as exemplary and/or illustrative and notrestrictive, the scope of the embodiments being indicated by theappended claims rather than by the foregoing description, and allalternate embodiments and changes to this embodiment which come withinthe meaning and range of equivalency of said claims are therefore to beembraced therein.

LOG OF REFERENCE NUMERALS

-   -   inspection module 100    -   microscope 105    -   X-Y stage platform 105 a    -   holder assembly 105 b    -   optical component 105 c    -   mount 105 d    -   holder assemblies 106 a-106 f    -   arms 107 a-107 f    -   CPU 110    -   computing module 200    -   database 205    -   processing module 210    -   pre-processing engine 210 a    -   AI vision solution engine 210 b    -   post-processing engine 210 c    -   user interface module 300    -   user interface 305    -   icon 306    -   interactive inspection icon 307    -   characteristic or visual indicia 307′, 307″    -   local database 310    -   database 320    -   icons 507, 507′, 507″    -   buttons 510, 515, 520

What is claimed is:
 1. A method of detecting defects in a rollingelement, the method comprising: (i) scanning an outer surface of arolling element via a microscope assembly to obtain a plurality ofsurface images of the rolling element; (ii) associating each of theplurality of surface images with a specific region of the rollingelement, wherein the plurality of surface images are stored in adatabase; (iii) identifying defects based on the plurality of surfaceimages via a processing module using an initial set of analysisparameters; (iv) generating a plurality of interactive inspection iconsassociated with the plurality of surface images that each correspond toa unique geographic region of the rolling element, and displaying theplurality of interactive inspection icons via a user interface, whereinthe user interface includes an input interface including an approveoption and a rejection option; (v) storing decisional informationregarding selection of the approve option and the rejection option fromthe input interface in the database; and (vi) updating the initial setof analysis parameters based on the decisional information.
 2. Themethod according to claim 1, wherein step (ii) further comprisesproviding the plurality of surface images to the database in real time.3. The method according to claim 1, wherein step (iii) further comprisesidentifying the defects via an AI vision solution engine that isconfigured to identify defects based on analysis parameters.
 4. Themethod according to claim 1, wherein the user interface is configured todisplay a two-dimensional image associated with the plurality of surfaceimages.
 5. The method according to claim 1, wherein the user interfaceis configured to display a three-dimensional image of the plurality ofsurface images.
 6. The method according to claim 1, wherein theplurality of interactive inspection icons are generated and displayed inreal-time while the microscope assembly scans the rolling element. 7.The method according to claim 1, wherein step (iii) includes identifyingat least one of: (i) a defect type on the outer surface of the rollingelement, or (ii) a size of a defect on the outer surface of the rollingelement.
 8. The method according to claim 1, wherein the plurality ofinteractive inspection icons are configured to include at least oneindicia that is representative of a defect condition.
 9. The methodaccording to claim 8, wherein the at least one indicia is configured tocorrespond to a level of severity of any defects on the outer surface ofthe rolling element in a specific geographic region.
 10. The methodaccording to claim 9, wherein at least one visual characteristic of theplurality of interactive inspection icons is configured to modifiedbased on the level of severity of any defects.
 11. The method accordingto claim 9, further comprising generating and displaying additionalinformation regarding the corresponding geographic region of the rollingelement via engagement of the plurality of interactive inspection iconson the user interface.
 12. The method according to claim 11, furthercomprising displaying at least one image of the corresponding geographicregion of the rolling element based on engagement of the plurality ofinteractive inspection icons on the user interface.
 13. The methodaccording to claim 1, further comprising generating a report includingthe surface images and the decisional information.
 14. The methodaccording to claim 1, wherein the microscope assembly incudes a mobilestage platform and a holder assembly including at least one armconfigured to engage a peripheral surface of the rolling element, suchthat the rolling element is configured to be rotated via the mobilestage platform of the microscope assembly.
 15. A method of detectingdefects in a rolling element, the method comprising: (i) scanning anouter surface of a rolling element via a microscope assembly to obtain aplurality of surface images of the rolling element; (ii) associatingeach of the plurality of surface images with a specific region of therolling element, wherein the plurality of surface images are stored in adatabase; (iii) identifying defects based on the plurality of surfaceimages via a processing module using an initial set of analysisparameters; (iv) generating a plurality of interactive inspection iconsassociated with the plurality of surface images that each correspond toa unique geographic region of the rolling element and displaying theplurality of interactive inspection icons via a user interface, whereinthe user interface includes an input interface including an approveoption and a rejection option, and the plurality of interactiveinspection icons are modified in real time with indicia associated withany defects identified during step (iii); (v) storing decisionalinformation regarding selection of the approve option and the rejectionoption from the input interface in the database; and (vi) updating theinitial set of analysis parameters based on the decisional information.16. The method according to claim 15, wherein step (iii) furthercomprises detecting a type of defect identified on the rolling element.17. The method according to claim 15, wherein the plurality ofinteractive inspection icons are initially a first color, and aremodified to display a second color to indicate a defect is detected or athird color to indicate a defect is not detected.
 18. The methodaccording to claim 15, wherein the user interface includes a pluralityof buttons, wherein a first button is configured to stop step (i), and asecond button is configured to display images obtained during step (i).19. The method according to claim 15, wherein at least 1,000 regions areidentified during step (ii).
 20. The method according to claim 15,further comprising calculating a depth and a width of any defectsidentified during step (iii).